Abstract

Background: Search filters are developed to improve the efficiency and effectiveness of searching and are created by identifying and combining search terms to retrieve records with a common feature (e.g. diagnostic accuracy). Objectives: To determine whether any existing methodological search filters designed to retrieve diagnostic test accuracy (DTA) studies perform well enough to be recommended for use in Cochrane DTA reviews. Methods: We conducted a systematic review of studies that developed and/or evaluated methodological filters designed to retrieve DTA studies in MEDLINE. Eligible studies had to report a measure of filter performance such as sensitivity or precision. Studies were identified through searching electronic databases and reference screening. Two reviewers independently screened search results. Inclusion assessment, data extraction and methodological assessment were conducted by one reviewer and checked by a second. Methodological assessment was undertaken using a modified checklist. Data to calculate sensitivity, specificity, precision and the number of abstracts needed to read (NNR) to identify one relevant study were extracted for each filter evaluation and were summarized in a forest plot. Results were analysed separately for studies that developed the original filters (development studies) and studies that evaluated existing filters (evaluation studies). Results: Fifteen studies (16 papers) reporting on 56 filters were included; 9 development studies, 6 evaluation studies, and two studies that both developed and evaluated filters (Figure 1). None of the filters had consistently high sensitivity and low NNR in the evaluation studies. Estimates of sensitivity ranged from 29% to 100% and NNRs, where available, ranged from 9 to 100. Estimates of sensitivity in the development studies were higher than those reported in the evaluation studies in 28 of 33 comparisons, and estimates of NNR were lower in 5 out of 10 comparisons. Conclusions: None of the filters evaluated showed sufficient sensitivity and precision to be recommended for identifying DTA studies in MEDLINE.
Hide

Abstract

BACKGROUND: Challenges exist when searching for diagnostic test accuracy (DTA) studies that include the design of DTA search strategies and selection of appropriate filters. This paper compares the performance of three MEDLINE search strategies for psychometric diagnostic test accuracy (DTA) studies in postnatal depression. METHODS: A reference set of six relevant studies was derived from a forward citation search via Web of Knowledge. The performance of the 'target condition and index test' method recommended by the Cochrane DTA Group was compared to two alternative strategies which included methodological filters. Outcome measures were total citations retrieved, sensitivity, precision and associated 95% confidence intervals (95%CI). RESULTS: The Cochrane recommended strategy and one of the filtered search strategies were equivalent in performance and both retrieved a total of 105 citations, sensitivity was 100% (95% CI 61%, 100%) and precision was 5.2% (2.6%, 11.9%). The second filtered search retrieved a total of 31 citations, sensitivity was 66.6% (30%, 90%) and precision was 12.9% (5.1%, 28.6%. This search missed the DTA study with most relevance to the DTA review. CONCLUSIONS: The Cochrane recommended search strategy, 'target condition and index test', method was pragmatic and sensitive. It was considered the optimum method for retrieval of relevant studies for a psychometric DTA review (in this case for postnatal depression). Potential limitations of using filtered searches during a psychometric mental health DTA review should be considered.
Hide

Abstract

OBJECTIVE: To compare the performance of MEDLINE searches using index test(s) and target condition (subject searches) with the same searches combined with methodological filters for test accuracy studies. STUDY DESIGN AND SETTING: We derived a reference set of 506 test accuracy studies indexed on MEDLINE from seven systematic reviews that conducted extensive searches. We compared the performance of "subject" with "filtered" searches (same searches combined with each of 22 filters). Outcome measures were number of reference set records missed, sensitivity, number needed to read (NNR), and precision (Number of reference set studies identified for every 100 records screened). RESULTS: Subject searches missed 47 of the 506 reference studies; filtered searches missed an additional 21 to 241 studies. Sensitivity was 91% for subject searches and ranged from 43% to 87% for filtered searches. The NNR was 56 (precision 2%) for subject searches and ranged from 7 to 51 (precision 2-15%) for filtered searches. CONCLUSIONS: Filtered searches miss additional studies compared with searches based on index test and target condition. None of the existing filters provided reductions in the NNR for acceptable sensitivity; currently available methodological filters should not be used to identify studies for inclusion in test accuracy reviews. Copyright © 2011 Elsevier Inc. All rights reserved.
Hide

Abstract

BACKGROUND: Systematic reviews of health care topics are valuable summaries of all pertinent studies on focused questions. However, finding all relevant primary studies for systematic reviews remains challenging. OBJECTIVES: To determine the performance of the Clinical Queries sensitive search filter for diagnostic accuracy studies for retrieving studies for systematic reviews. METHODS: We compared the yield of the sensitive Clinical Queries diagnosis search filter for MEDLINE and EMBASE to retrieve studies in diagnostic accuracy systematic reviews reported in ACP Journal Club in 2006. RESULTS: Twelve of 22 diagnostic accuracy reviews (452 included studies) met the inclusion criteria. After excluding 11 studies not in MEDLINE or EMBASE, 95% of articles (417 of 441) were captured by the sensitive Clinical Queries diagnosis search filter (MEDLINE and EMBASE combined). Of 24 studies not retrieved by the filter, 22 were not diagnostic accuracy studies. Reanalysis of the Clinical Queries filter without these 22 nondiagnosis articles increased its performance to 99% (417 of 419). We found no substantive impact of the two articles missed by the Clinical Queries filter on the conclusions of the systematic reviews in which they were cited. CONCLUSION: The sensitive Clinical Queries diagnostic search filter captured 99% of articles and 100% of substantive articles indexed in MEDLINE and EMBASE in diagnostic accuracy systematic reviews.
Hide

Abstract

OBJECTIVE: To determine the usefulness of methodological filters in search strategies for diagnostic studies in systematic reviews. STUDY DESIGN AND SETTING: We made an inventory of existing methodological search filters for diagnostic accuracy studies and applied them in PubMed to a reference set derived from 27 published systematic reviews in a broad range of clinical fields. Outcome measures were the fraction of not identified relevant studies and the reduction in the number of studies to read. RESULTS: We tested 12 search filters. Of the studies included in the systematic reviews, 2%-28% did not pass the sensitive search filters, 4%-24% did not pass the accurate filters, and 39%-42% did not pass the specific filters. Decrease in number-needed-to-read when a search filter was used in a search strategy for a diagnostic systematic review varied from 0% to 77%. CONCLUSION: The use of methodological filters to identify diagnostic accuracy studies can lead to omission of a considerable number of relevant studies that would otherwise be included. When preparing a systematic review, it may be preferable to avoid using methodological filters.
Hide

Abstract

OBJECTIVE: To develop and test a Medline filter that allows clinicians to search for articles within a clinical discipline, rather than searching the entire Medline database. DESIGN: Diagnostic test assessment framework with development and validation phases. SETTING: Sample of 4657 articles published in 2006 from 40 journals. Reviews Each article was manually reviewed, and 19.8% contained information relevant to the discipline of nephrology. The performance of 1 155 087 unique renal filters was compared with the manual review. MAIN OUTCOME MEASURES: Sensitivity, specificity, precision, and accuracy of each filter. RESULTS: The best renal filters combined two to 14 terms or phrases and included the terms "kidney" with multiple endings (that is, truncation), "renal replacement therapy", "renal dialysis", "kidney function tests", "renal", "nephr" truncated, "glomerul" truncated, and "proteinuria". These filters achieved peak sensitivities of 97.8% and specificities of 98.5%. Performance of filters remained excellent in the validation phase. CONCLUSIONS: Medline can be filtered for the discipline of nephrology in a reliable manner. Storing these high performance renal filters in PubMed could help clinicians with their everyday searching. Filters can also be developed for other clinical disciplines by using similar methods.
Hide

Abstract

OBJECTIVES: To assess the performance of published search filters in finding diagnostic test accuracy studies. METHODS: Diagnostic test accuracy search filters were identified by searching medline, our own files and by requesting unpublished filters from colleagues. We applied the filters to a case study review of diagnostic test accuracy studies for urinary tract infections (UTI) in young children. The included studies with records in medline formed the gold standard. The performance of the filters in finding those gold standard records was assessed. RESULTS: We identified twenty-three diagnostic test accuracy search filters for use with medline. The case study systematic review of UTI included 179 studies of diagnostic test accuracy, of which 160 were available in medline. The filters showed a wide range of sensitivities (range: 20.6% to 86.9%) and precision (range: 1% to 9.4%). CONCLUSIONS: Our results broadly support those reported in two other studies. The search filters tested do not offer an adequate trade-off between sensitivity and precision to be used to identify studies for systematic reviews. However, there are methods available to explore whether search filters are viable based on an objective statistical analysis of the text and indexing used in records.
Hide

Abstract

OBJECTIVE: To develop and test optimal Medline search strategies for retrieving sound clinical studies on prevention or treatment of health disorders. DESIGN: Analytical survey. DATA SOURCES: 161 clinical journals indexed in Medline for the year 2000. MAIN OUTCOME MEASURES: Sensitivity, specificity, precision, and accuracy of 4862 unique terms in 18 404 combinations. RESULTS: Only 1587 (24.2%) of 6568 articles on treatment met criteria for testing clinical interventions. Combinations of search terms reached peak sensitivities of 99.3% (95% confidence interval 98.7% to 99.8%) at a specificity of 70.4% (69.8% to 70.9%). Compared with best single terms, best multiple terms increased sensitivity for sound studies by 4.1% (absolute increase), but with substantial loss of specificity (absolute difference 23.7%) when sensitivity was maximised. When terms were combined to maximise specificity, 97.4% (97.3% to 97.6%) was achieved, about the same as that achieved by the best single term (97.6%, 97.4% to 97.7%). The strategies newly reported in this paper outperformed other validated search strategies except for two strategies that had slightly higher specificity (98.1% and 97.6% v 97.4%) but lower sensitivity (42.0% and 92.8% v 93.1%). CONCLUSION: New empirical search strategies have been validated to optimise retrieval from Medline of articles reporting high quality clinical studies on prevention or treatment of health disorders.
Hide